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Abstract 

A common report from anecdotal writing over many generations of educators is that it is the teacher 
who usually learns the most during the process of gathering content materials, designing, teaching 
and evaluating student performance. In this project we address this issue by developing an innovative 
instructional design in which collaborative groups of students working at distance create, share and 
assess learning content (in the form of learning objects) with their peers through online learning 
portals. The results of this process are assessed via surveys, discussions, reflective essays and peer 
evaluations. We conclude that instructional models based upon student construction of content and 
orchestration of learning activities can reduce instructor workload, provide opportunity for students to 
acquire new skills while increasing their subject content knowledge, and create a lasting legacy of re- 
usable learning objects. 

Introduction 

How can today's educators successfully mesh the education and training demands of a rapidly 
changing global knowledge society, the socioeconomic constraints of students, institutes, and 
governments and technocentric, consumer orientated student body? One answer lies in the creation 
of innovative, flexible instructional course designs aimed at creating active learning communities in 
which the students take on the major roles of constructing, sharing and teaching the course content. 
In this paper we detail such a design and provide descriptive evaluation data obtained from surveys of 
students' perception, reflective essays and our own perceptions of the quality and costs of developing 
and teaching a graduate course using this design. 

This action research project addresses the research question - what are the social, pedagogical, 
economic and communication technology factors that are most critical for satisfaction, high 
perceptions of learning and cost effective development to students engaged in collaborative, project 
based learning activities in a graduate level educational course? 

Literature Review 

The movement from the industrial to the information age challenges traditional institutes and the 
pedagogies that they uphold. Traditional institutes produce learners who "...are not asked to take 
responsibility for their own learning - they do not set learning goals, ask questions to direct learning 


activities, assess their learning strategies and approaches, or reflect on what they have learned" 
(Grabinger and Dunlap, 2001: 1). According to the American National Science Foundation, these 
institutes produce workers who 'go out into the workforce ill-prepared to solve real problems in a 
cooperative way, lacking the skills and motivation to continue learning" (p. iii). This lack of 
involvement in the design and implementation of their own learning results in low levels of 
'metacognition' or the skill of learning how to monitor their own thinking and learning processes, and 
the ability to transfer knowledge from one situation to the next (Grabinger & Dunlap, 2001: 2). Global 
economy demands a new kind of worker - one that is flexible, autonomous and group oriented, 
possesses critical thinking skills, can solve a range of problems, is capable of multitasking and is 
committed to lifelong learning. (Keegan, 1996; Collis and Moonen, 2001, Grabinger and Dunlap, 
2001; Springer, Stanne and Donovan, 1997.) 

Educators need a pedagogy, or instructional theory, upon which to construct their courses (Seels, 
1997). This provides a foundation from which all aspects of course design evolve. Like many others 
developing collaborative learning models for online delivery (Duffy & Kirkley, 2004; Campbell, 2003), 
the MDE 663 course design is based on a constructivist pedagogy. "Constructivists argue that 
knowledge is not so much discovered, or transmitted intact from one person to another, as it is 
created or 'constructed' by individuals attempting to bring meaning and coherence to new information 
and to integrate this knowledge with their previous experience." (Rourke and Anderson, 2001: 2.) 
Inherent in this definition is the idea that learners need to interact with others by sharing, debating and 
discussing ideas "...to integrate and elaborate knowledge in ways that facilitate higher-order learning." 
(p. 3.) 


A core theme to a constructivist instructional theory is 'learner-centeredness', meaning that courses 
are designed with learner attributes and choice in mind. Collis and Moonen, 2001, propose a 
'flexibility and activity' framework in which they define traditional, existing and future instructional 
designs by placing them in one of four 'Quadrants' based largely upon the degree of learner- 
centeredness. Traditional instructional designs of Quadrant I are characterized as being less flexible 
(i.e. having less student choice regarding time, content, instructional activities, resources, delivery and 
logistics) and having activities that primarily center around independent knowledge acquisition. 
Quadrant II course designs are similarly less flexible: student participatory activities exist, but are 
primarily structured by instructors. More student choice is afforded in Quadrant III, but the main 
activity usually remains focused on independent knowledge acquisition. Quadrant IV exemplifies ideal 
or future instructional designs, which are flexible and are based on participatory or primarily 
contribution-oriented activities. 
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Figure 1 Flexibility - Activity Framework - Collis and Moonen (2001 : 24) 







Other constructivist authors build their instructional designs around a model of active student learning 
as well. For example, Sfard (1998) describes a 'participation-oriented' model. Dopper and Dijkman 
(1997) and Simons (1999) present theirs as 'action learning'. Kearsley and Shneiderman (1998) 
capture active learning via their 'engagement theory'. Collis and Moonen, 2001, define their model in 
terms of 'the contributing student'. Grabinger and Dunlap, 2001, propose the concept of 'Rich 
Environments for Active Learning' (or 'REALs'), based on 'intentional learning'. 

Clearly, constructivist designs maximize the amount of active learning in which students are engaged. 
How can educators "engage learners in dynamic, authentic learning activities that increase their 
control and responsibility over the learning process while they learn problem-solving and collaborative 
skills and content" (Grabinger and Dunlap, 2001:2)? Grabinger and Dunlap's REALs (2001: 2) offer 5 
instructional strategies that they believe generate such learners: 

• Intentional learning and student responsibility 

• Authentic contexts and relevant, meaningful learning 

• Dynamic, generative learning activities 

• Collaboration and social negotiation of meaning, and 

• Extensive reflection and self-assessment 


Once educators have developed instructional strategies that revolve around active student learning, 
they must decide what content and media resources to use. Contrary to normal practice in which a 
major task of the instructor is to create or select content for study, Collis and Moonen, 2001, argue 
that there should be little, if any, instructor-developed content included in courses based on an active 
(or 'participatory') student learning model. Most content is researched and/or generated, synthesized, 
presented, discussed and evaluated by the students themselves. Instead of focusing on content, the 
instructor works to develop communication strategies that enable students to find, present and share 
information and thus accumulate knowledge in an effective manner. Therefore, attention must also 
be paid to communication and collaborative workspace technology choices. 

Putting all of these variables together creates a complex, dynamic learning environment where 
instructors are no longer patriarchs (Springer, Stanne and Donovan, 1997), but mentors and active 
learning role models. Students are no longer passive recipients of knowledge, but contributing 
members of a learning community. Net-based technologies offer a myriad of communication and 
collaboration options designed to support knowledge building communities, while allowing participants 
to shift time and space constraints. Is it possible to use these technologies and the pedagogical 
insights above to create cost and learning effective courses? 

Method 
Case Setting 

This study examines data from the experience and output of 17 Athabasca University (AU) graduate 
students enrolled in the Masters of Distance Education MDE 663: Emerging issues in educational 
technology during the 2003 fall semester. The 13-week course was broken into three sections - a 
series of introductory lecture/discussions presented by the instructor and guest speakers (6 weeks), 
weekly 'Open Houses' at the team portal sites (4 weeks) and an assortment of reflection and 
concluding activities (3 weeks). The learning management system (LMS) used was WebCT, version 
3.1 . Two-hour synchronous audio-graphic sessions supported by the net-based Elluminate vClass (or 
virtual classroom) were offered once a week. The main student assignment during the term was the 
creation of a REAL or 'learning portal' that focused on one emerging issue related to educational 
technology. The term ‘portal’ has a number of definitions and connotations, many associated with net 
based marketing. We use the term to mean an entry point to a multidimensional educational site used 



for exploration, information and learning activities. The students developed these portals as multi-level 
learning objects to share and teach their colleagues during a one-week ‘Open House’. The students 
were given opportunity to use any technologies they wished to support small team (i.e. 4 or 5 people) 
collaboration, create portals and host their 'Open House'. Technologies used by portal teams 
included Blogs, Learning Management Systems, Content Management Systems, custom web sites, 
quiz and game generators, and asynchronous and synchronous chats and forums. 


Then topics chosen for the 2003 version of the course were chosen by the instructor, based upon 
their currency, potential educational impact and high interest. They were: 

1 . Educational semantic web 

2. Open Source distribution 

3. E-learning Standards 

4. Educational objects and repositories 


The assessment of the course consisted of both a collaborative (all team members received the same 
grade) and individual assessment. An instructor evaluation of the team portal was used to calculate 
40% of the student grade. Individual assignments include answering a comprehensive question 
presented by other portal teams, evaluating other portals and reflecting on the individual and group 
learning process. 



Figure 2 Schedule of learning activities 


Data Collection and Analysis 


This action research project used case study methodology involving five major instruments to gain an 
understanding of the learners' interpretation and evaluation of the process and technologies used in 
the course. These instruments included: 

1. A survey detailing initial student skills and interests - completed voluntarily, not graded, and 
used to determine portal group memberships, 

2. A 1,000 word reflection essay on the course that included an evaluation of personal and team 
members' contributions and learning experiences, graded by the instructor, 

3. An individual and group evaluation form to accompany the reflection essay, 

4. A questionnaire completed after the course is finished and grades are distributed, not graded 
and completed voluntarily, and 

5. Research/participant/instructor insights gained through participation in the course as teacher 
and teaching assistant. 


Each of these instruments is described in further detail below. 



Group Interest and Skills Survey 


The group interest and skills survey was used at the beginning of the semester to help the instructor 
place students into portal teams. Students were asked to assess personal interest and expertise 
needed to create each portal project. The results of this survey were used to create portal teams and 
to compare students' self-perceptions of topic interest and individual group role skills at the beginning 
of the semester to their term end reflection papers, questionnaire and instructor perceptions. 

Reflection Essay 

Each student was required to submit a 1,000-word reflection essay at the conclusion of the course. 
The data from the reflection essay was synthesized into four broad areas of discussion: individual 
learning, group learning, technology issues and efficacy of the course pedagogy. The results were 
used to examine growth in student learning, metacognition, group dynamics, technological concerns 
and issues, and perceptions of the course's design. These results were triangulated with the other 
research tools to ensure a greater degree of accuracy in interpretation. 

Individual and Group Evaluation Form 

The individual and group evaluation form was part of the reflection assignment. It was broken into 
three broad areas: fulfilling individual roles (using the role titles and descriptions established by the 
instructor at the beginning of the term), timely completion of tasks and general group functioning. 
Each student was asked to evaluate their own performance, as well as that of the other members in 
their portal group. This form assigned a Likert scale of 1 - 10, with 1 showing little or no evidence of 
contribution, 5 indicating average contribution and 10 demonstrating exceptional contribution. The 
results were used to compare individual reflection essays to Likert scale scores assigned in the 
individual/group evaluation form to verify consistency between the two documents in terms of 
individual, group, technological and pedagogical issues. These findings were used to compare 
student perceptions and skills over the course of the term to ensure accuracy in research 
interpretations. 

Term End Questionnaire 

Students voluntarily completed the term end online questionnaire (posted at: 
http://survey.icaap.org/html/mde663.htm with final results at: http://survey.icaap.org/html/results.htm), 
after the course was finished and they had been informed of their grades in order to meet ethics 
review guidelines for informed, non-coercive consent. It was broken into 5 main sections: background 
information; social, pedagogical and technological factors; and a conclusion. This questionnaire 
consisted of Likert scale questions and anecdotal comments. The results were used to record and 
compare student perceptions and evidence of student learning, metacognition, group dynamics, 
technology choices and course pedagogy throughout the term. 

Instructor/Researcher Insights 

The instructor and teaching assistant gained many insights as the course evolved from an idea to its 
first implementation. Student development, difficulties and recommendations enabled the instructor to 
not only evaluate evidence of student understanding and performance, but also the structure of the 
course itself. Reflection and metacognition at the instructor level was enhanced by subsequent 
research findings. The results of these discoveries were used to re-evaluate and revise various 
components of the course's design, and to subjectively assess the value of the underlying pedagogy. 

Results 

Group Interest and Skills Survey 



The group interest and skills survey allowed the instructor to see which topic of four chosen by the 
instructor was of most interest to students. Students also indicated in which of the five roles (i.e. 
project manager, webmaster, instructional designer, graphics developer and resource gatherer) they 
had greatest interest and skill. A number of students said they had careers that required them to play 
one or more of these roles in daily life. Interestingly, some students choose roles and topics with 
which they had considerable expertise and background knowledge already; others specifically choose 
unfamiliar territory to develop new knowledge and skills. 

Reflection Essay 

Analysis of the student reflection essays provided a wealth of data related to their experience of the 
course. Suggested guidelines for the essay requested students to comment on personal learning, 
technology issues and various collaborative and individual activities in which they participated. 

Most students reported positively on their personal learning during the course. One commented: "At 
one point, I wrote to my colleagues stating that there is no one who has learned more than me in this 
course and I mean it. I learned so much in this time period that my learning curve was almost 
vertical." For many the content area was new and exciting, though a few reported problems in 
following all discussion and activities due to their limited background in the subject area. 

Group process issues revolved around shared purposes, goals and outcomes, co-operative 
behaviour/interdependence (e.g. trust, planning, support), and individual accountability. As expected, 
most students noted that group cohesion developed through the process of portal creation. The 
equitable sharing of workload created some problems but also increased group empathy for absent 
members or those with personal constraints on their capacity to participate. Only one student felt that 
absences and unequal work distribution eroded group cohesion. Three students reported positive 
group collaboration efforts, nine noted both advantages and disadvantages of group process, and 
seven reported group capitalization on each others' experiences and strengths, expressing 
appreciation of each other's differences. Individual accountability was a big issue to some. The 
portals were generally completed by dividing tasks between different members and assembling 
finished components into an integrative whole (jigsaw style). Since this model emphasized individual 
accountability, it allowed the groups to progress without need to negotiate all content. However, this 
strategy likely limited the in-depth discussion and critique that results with more collaborative 
development models. 

All but one of the students spoke positively about the course pedagogy, with one reporting, 

"The learning design proved effective in terms of team members' construction of knowledge and 
project deliverables for several reasons. First, course expectations and assessments were clearly 
defined (sense of community was a valued outcome). Second, user-friendly tools (communication, 
Web design) and hardware were available. Third, mature, self-directed, experienced MDE program 
graduate students performed an authentic exercise. Fourth, the team participated fully." 

The unhappy student felt that the content was too advanced and suggested that readings should be 
available before the course began. She also reported serious communication technology problems. 
Another student suggested that instructor-imposed deadlines for partial project completion might keep 
groups more focused, and therefore, more organized and cohesive. 

Individual and Group Evaluation Survey 

All students submitted an individual and group evaluation survey with their reflection essay. The 
results indicated that two of the groups had assigned roles as suggested by the literature provided by 
the instructor and used in creating the portal groups from the interest and skills survey at the 
beginning of the term. Another group 'self-organized', allowing members to change roles based on 



time, expertise and available resources because they believed that this facilitated group consensus 
and coherence. The final group developed new roles for team members. 

As in many course with heterogeneous groups of students, their perception of value of the course was 
not homogeneous. Two students felt the course was dry, and that the content too difficult to absorb 
or assimilate. They had technological and time management problems, which they felt were 
instructional design issues. They were resentful of the time they had to spend online, and were 
particularly unhappy with the weekly synchronous sessions. One reported family time conflicts. 
Other group members saw things differently, pointing out these individuals' lack of attendance at 
meetings, inattention to group emails, and irresponsibility in getting tasks completed on time and in a 
scholarly manner. 

These discrepancies point to the problem of using self-evaluations for assigning performance marks 
for the course. Many high performance and successful students are suspicious of design formats that 
force them to rely on other students who may not share their commitment, organization style, 
aptitudes or aspirations for high marks. These students are often reluctant to volunteer for 
collaborative work, however, such collaboration is inherent in much work required in the distributed, 
global workplaces common to many professions. Thus, we feel that even forced participation is a 
valid component of a high quality master’s degree experience 

Term End Questionnaire 

Thirteen of the seventeen MDE 663 students (or 76%) responded anonymously to the term end 
questionnaire. The questionnaire results are available at http://survey.icaap.org/html/results.htm. The 
questionnaire addressed a number of social participation factors in the course. The students indicated 
a slightly greater sense of belonging to the MDE 663 class than most other MDE classes. Most 
disagreed that it would have been easier to decide portal group roles had they been in a face-to-face 
setting. They were slightly pleased that the instructor picked groups and most felt that the workload 
had been shared equitably amongst team members. Although students expressed satisfaction with 
the group process, few actively planned on taking courses or communicating further with other portal 
group members as a result of friendships made during the process. 

Perhaps not surprisingly (given the overwhelming predominance of individual learning and 
assessment in their current and past educational programs) students generally felt that they learned 
slightly better and achieved slightly better grades when working independently. They had very mixed 
views on questions comparing the workloads of independent study, and small and large group work. 
This probably reflects the variation in amount of group work performed based upon individual skill 
level, interest and commitment. 

When asked to compare this course to others in the program, students indicated that the workload 
was slightly greater, but that the amount of learning was about the same. Perhaps it is not surprising 
that given the novelty and challenges of communicating at a distance, 62% of students found the 
course more stressful while only 8% found it less stressful than other courses in the program. Despite 
this, 62% indicated that they would take another course offered using this collaborative framework, 
whereas only 16% would not. It is important to note that these students are studying the theory and 
practice of distance education, so their level of expertise and interest in novel pedagogies and 
technologies is likely to be greater than that of students from other disciplines. 

The questionnaire's next section looked at communication and information technologies used within 
portal groups. The technologies perceived as most useful for communications were the synchronous 
audiographic technologies of vClass (used for both full class and informal team meetings) and the 
ubiquitous asynchronous email. Here is one such comment, 



"Synchronous component aided in the development of relationships and learning community. 
Audio, for whatever reason, was a significant factor in the development of said relationships." 


And 


"The Monday night synchronous sessions were very necessary. They helped to bond the 
class, and put a "face" to many of the participants. The lack of face to face was not a detriment. 
Personalities and abilities could be discerned without F2F classroom physical interaction. 
Oddly enough, while I enjoyed seeing people's pictures (on their portal) I found it put barriers 
on my personal interpretation of them. I think physical anonymity allowed for a greater freedom 
in the social dynamics. If I were in a classroom deciding who to work with perhaps I would have 
gravitated toward someone of my own age, or someone that looked like they would be fun to 
work with, or pleasant to observe." 

In sharp contrast, here is one respondent's view: 

"Found I preferred the asynchronous to the virtual class. Did not like having to dedicate the 
time to attend class, plus read the asynchronous postings and give input to them as well as 
complete all the readings and assignments. Found that the synchronous participation was over 
and above the time spent with other classes with asynchronous CMC. This was because, as I 
mentioned, this course also included the asynchronous CMC as well. If offered the option of 
taking another course with a virtual class included, I would decline it." 


Communication Tool 

Frequency (Mean) 

Private email 

4.9 

Illuminate vClass 

4.8 

Group/CMC workspace 

3.1 

Synchronous chat tool (e.g. Yahoo Messenger, 
PalTalk) 

1.9 

One - to - one telephone 

1.2 

Teleconferencing (via telephone) 

1.0 

Other communication tool (please specify) 

1.0 


Key: 

1 = never 2 = rarely 3 = occasionally 4 = frequently 5 = extensively 
Table 1 : Section Dill a - Frequency of communication tools use 

The course contained a variety of learning activities and features. These are summarized below, 
along with the perception of value attributed to them by students. 


Tool 

Description 

Mean 

Other 

portals 

WWW based site developed by three other small groups in 
course, containing organizing map, links to information 
sources, overview of major issues, asynchronous 
discussion site, and a variety of learning games, activities, 
etc 

4.4 

Course 

syllabus 

Introduction and overview of instructor contact information, 
course philosophy, objectives, description, and structure, 
student and technology prerequisites, creating student 
teams, calendar and evaluation information -found on class 
WebCT home page. 

4.1 

Assigned 

readings 

Readings on course philosophy, goals, introduction, format 
and group dynamics/roles, assignments and evaluation 
criteria; readings on 4 content topics - found on class 

4.0 



WebCT site. 


Own portal 
team 

4 portal teams, each focusing on one of the 4 topics:. 
Three teams had 4 student members; one had 5. 

4.0 

Introductor 
y Essays 

4 instructor-written essays, one on each content topic 
(above) - found on WebCT site. 

3.9 

Reflection 
and self 
assessme 
nt 

assignmen 

t 

Individual assignment consisting of 1 000 word essay 
focusing on student perceptions of individual learning, 
group dynamics, technological and pedagogical issues. 
Includes a survey with 3 sections: fulfilling individual roles, 
timely completion of tasks and contribution to group 
functioning, that each student rates themselves and other 
members of the group by - criteria and survey found on 
WebCT site. (10% of final grade.) 

3.9 

Optional 

readings 

Four sets of readings (one on each content topic) 
presented as annotated reference lists with hyperlinks to 
original online documents - found on WebCT site 

3.9 

Course 

calendar 

Class term long day calendar with 'public' meeting, 
assignment and related dates/comments/links posted by 
professor; also allows students to post private dates - 
found on WebCT site. 

3.8 

Portal 

evaluation 

s 

1 000 word individual assignment critique giving overview 
of technological strengths/weaknesses, site design and 
most effective learning components of other 3 portals (20% 
of final grade). 

3.8 

vClass 

Illuminate 'virtual classroom' with synchronous text, audio, 
whiteboard, application sharing and video options offered 
via Internet. Real-time 2 hour class sessions began at 
18:00 MST on 12 Mondays of the term. Text and 
whiteboard components can be individually saved; all 
sessions archived via website link. 

3.7 

Portal 

Questions 

Individual 400-600 word assignment asking students to 
answer a group-generated question on some major factor 
from each of 3 other portals (graded by professor and 
worth 30% of final grade). 

3.6 

Discussion 

Board 

WebCT asynchronous discussion site that allows individual 
'threaded' text-based comments (and a variety of file 
attachments) to be posted under general topic headings, 
such as the week's content issues, technical problems, etc. 

3.0 

Video 

introductio 

n 

Introduction of instructor's background, interests and 
research interests, linked to instructor's course introduction 
and found on WebCT site. 

2.8 

Student 

homepage 

s 

Student-generated homepages introducing each student to 
the class, using WebCT web authoring tools and posted to 
WebCT site homepage. 

2.4 


Key: 

1=not useful, 2=rarely useful, 3=occasionally useful, 4=often useful, 5=very useful 


Table 2 : Section Dill b - Value of communication and learning tools 

The questionnaire concluded with an opportunity to make comments about any aspect of the 
course and/or research project. Comments by seven respondents showed a very positive feeling 




about the learning quality and relationship between students and the instructor, and within the portal 
teams. A member of one team stated, 

"Our portal group displayed extraordinary cohesion. It was interesting how other portal group 
members were curious about our team process and success. All our team seemed empowered 
throughout the group process and were quite thrilled to share the magic that was apparent." 

This respondent's sentiment is echoed in another's statement as well, 

"I keep thinking and periodically writing about this course - "what a great way to learn!" I have 
taught in adult education for 25 years and what my philosophy about learning and teaching has 
become - I want to develop education that stimulates and challenges "me" because I am the worst 
kind of student - one that does not want to learn! This course - I will always remember because it 
made me want to learn! And the learning seemed to come easier than in other courses." 

Specific attributes of the course most appreciated included the equal attention paid to 
pedagogical and social aspects of learning, appropriate use and exposure to a variety of 
current educational technology learning methods, capacity to use the model in their own 
classroom, and amount of student autonomy and flexibility this course offered. 

Three students confessed that they liked learning independently more than they did in groups, 
although one thought they had learned a lot in this class. One student said, 

"I think I am an extreme when it comes to working independently. I am more intrigued by 
course reading than interacting with other people. However, in saying that, I always read the 
postings on the asynchronous discussions and I seldom missed a synchronous 
session. ..hoping to glean some new ways of thinking about things." 

Discussion 

One means of evaluating a course design is to assess its construct validity by comparing it to the 
theoretical model upon which the design is based. Grabinger and Dunlap, 2001, offer a model for 
such an evaluation based upon their REALs. This model suggests students should practice taking 
ownership over their own learning. "To be intentional learners, students must identify learning 
deficiencies and strengths, make and implement plans, develop metacognitive awareness, and revise 
those plans and actions based on that awareness" (p. 3). The MDE 663 term begins with the 
interest/skills survey, a tool meant not only to help develop balanced groups, but to help students 
evaluate their interests and abilities. Course and project goals are presented on the WebCT site, as 
well as in the first and last synchronous sessions. The group project's aim is to produce a portal on an 
emerging educational technology issue where a WWW-based audience of educators could deepen 
their understanding of new educational technology applications. Students evaluate each other’s sites 
in order to develop their critical thinking skills and to insure that they spend time learning the content 
by participating in activities presented by the other teams. At the end of the term they write a 
reflection paper, detailing their awareness of individual and group learning, technology issues and 
course pedagogy. The instructor does not use the self and group evaluation for student assessment 
purposes. Rather the self-assessment is used to enhance metacognition - forcing students to assess 
their own performance, that of other individuals and the team as whole. A danger in assigning course 
credit for this evaluation is the challenge students have in reliably rating each other and themselves in 
the absence of clear criteria and an assessment rubric. 

Students report mixed feelings on benefits of participating in group activities. Overall, they 
characterize themselves as learning slightly better individually than in small groups. Sixty percent of 
their grades come from individual assignment submissions, but all assignments are generated from 
group portal activities. Only one of four groups appears to exemplify a strong, positive learning 
community throughout the course. The other three indicate periods of time when communication 



breaks down, resulting in frustration. The capacity for teams to empower leaders to emerge and 
initiate activities to resolve communications breakdowns seems critical to effective group process. 

Constructivist and situated knowledge theorists argue that students need to work on realistic 
problems that hold relevance to their daily lives. Such problems help them to develop richer, deeper 
knowledge structures that are more likely to be transferred to new situations. "Ill structured, complex 
problems require a team approach that provides natural opportunities for learners to seek out 
information, test and refine their ideas, and help each other understand the context" (Grabinger and 
Dunlap, 2001: 3). This encourages interaction, collaboration and social negotiation of meaning. MDE 
663 is based on four complex emerging issues that are likely to have major impact on the 
development of distance and distributed learning systems. The course design expects students to 
use whatever project-based, collaborative and presentation technologies they can to produce a portal 
that is cohesive, user-friendly, interactive and results in effective learning. The reader is invited to 
assess the success of the portals in achieving these goals by visiting the portals at 
http://cde.athabascau.ca. 

'Dynamic, generative learning activities' (Grabinger and Dunlap, 2001) require projects that are based 
in an authentic context, use multimedia, allow for experimentation and testing and encourage 
research. MDE 663's unique structure enables each term's content to change as newly emerging 
educational technology issues arise. The collaborative work helps students develop social skills in 
online environments. Grabinger and Dunlap's final requirement is reflection and self-assessment. The 
MDE 663 design requires students to individually reflect on course content, processes, tools and 
resources. Thus, the real assessors of learning are the students, because they evaluate each other's 
projects as well as reflect on their own group and individual progress. 

Collis and Moonen (2001 : 1 1 9) list a number of criteria and suggestions for courses that exemplify the 
pedagogical principles of a Quadrant IV course. Comparing this list to the MDE 663 design in the 
table below reveals that the MDE 663 course design virtually mirrors their ideal course structure. The 
only exception is that the WebCT site does not currently sport a 'frequently asked que stions' section. 


Course 

component 

Collis and Moonen's 

ideal WWW-based course site 

details 

MDE 663 

course component details 

Course 

organizatio 

n 

-all course information, updates, 
emails, student progress, student 
assignments submissions available 
on one site; ideally accessible via the 
Internet 

-all course information, updates, 
emails, student grades/progress 
and course calendar available on 
WebCT site - accessible via the 
Internet 

Lectures, 

contact 

sessions 

- lecture notes and related resources 
available on same site. 

- audio, text and whiteboard 
presentations from the weekly 
vClass sessions archived on 
website, PowerPoint presentations 
and follow-up comments/further 
information available on WebCT 
discussion site 

Self-study, 

exercises 

-professor and students can add 
further study materials to site; 
professor should be able to access 
site to retrieve student assignments, 
give feedback and record grades in 
one WWW environment. 

-students can add materials and 
links via the asynchronous 
discussion site, the student 
presentation site and their 
homepages on WebCT site 
-a link to student portal sites is on 
the WebCT homepage, but other 
assignments are submitted by 
private email 


Course 

component 

Collis and Moonen's 

ideal WWW-based course site 

details 

MDE 663 

course component details 



-grades accessible by professor 
and students on WebCT site 

Major 

assignment 

-expectation details posted on site, 
as well as links to previous years' 
student assignments saves time for 
all by reducing questions, etc. 

-WebCT site lists details and 
grading criteria for all assignments 
-new cohorts of MDE 663 students 
will be able to access previous 
years' portal projects via the 
WebCT site link, as well as AU's 
site. 

Mentoring, 
communica 
tion not 

specific to 
the above 
component 
s 

-frequently asked questions (FAQ's) 
can save new cohorts of students 
and professor time 

-mentoring and instructor contact 
with students allows instructor to 
answer questions when convenient, 
record them for future reference, 
forward them and remain in contact, 
even when the professor or students 
are out of town. 

-does not currently have a FAQ 
section on site 

-WebCT site has archival 
asynchronous discussion site, 
which includes an administrative 
subsection, technical 

support/discussion site and both 
private (i.e. portal group only) and 
public topic discussion sections. 

Table 3: Co 

lis and Moonen's , 2001, Ideal WWW-based course site vs. MDE 663 


Conclusion 

The experience of designing, building, facilitating, and evaluating this course produces a host of 
tentative conclusions, lessons learned, recommendations and suggestions for further developmental 
research. 

From the review of the literature, and now confirmed by our own experience in this course, we 
conclude that an instructional design based on student construction and sharing of knowledge in a 
web based portal is perceived to be a valued and worthwhile learning experience. Students report this 
design provides as much or more learning, challenge and skill acquisition as other courses in the 
program do. This is consistent with findings by Springer, Stanne and Donovan, 1997, who report that, 
"the main effect of small-group learning on achievement, persistence, and attitudes among 
undergraduates in Science, Mathematics, Engineering and Technology (SMET) was significant and 
positive'. They base this conclusion on a metanalysis of 383 research studies of collaborative projects 
engaging SMET students from 1980 to 1996. Given this positive assessment of learning 
effectiveness, we can begin to look at learning efficiency by examining the cost of the course. 

Unlike many other web-based designs for relatively small numbers of learners, this design is cost 
effective based on three major features. First, the time to create the course is modest. The use of a 
full-featured LMS provides a platform for relatively easy display of content, facilitation of discourse, 
scheduling of activities and tracking of student progress. This design can easily be re-used during 
subsequent courses. Changing the design to a different set of issues in this or another discipline 
requires a modest amount of preparation. This includes development of: 

• introductory content for each of the ‘issues' created as lecture content (print or canned 
audio), 

• a list of compulsory and optional readings, web sites or other learning resources; and 


• design and facilitation of a series of asynchronous or synchronous learning activities, 
discussions, guest speakers, etc. to serve as content during the four introductory weeks 
of the course. 

Secondly, the course has potential to save development costs in other courses because the legacy 
portals are re-usable learning objects. On the final week of class, the students worked in teams to 
'metatag' and enter their portals into three major learning portals (Adlib, CAREO and Merlot), thus 
facilitating their re-use not only within our program, but also by teachers and students globally. The 
portals can be accessed and enhanced in subsequent courses, where these subjects may be refined 
and explored in greater detail. They can also be used in courses for which the portal content is only of 
minor use by informal learners investigating important emerging topics on their own volition, and as 
models and exemplars for subsequent courses' teams using a similar instructional design. 

Thirdly, the course design, with its emphasis on student management of learning activities during the 
final weeks of the course, reduces instructor time commitments. The instructor’s role during the final 
six weeks of the course becomes one of a supportive class participant, allowing reallocation of time to 
assessment activities which increase (as is common in most formal education courses) near the end 
of the course. 

There are also a number of important lessons learned that we will use in subsequent courses. We 
must remain cognizant of the vast differences in student backgrounds and experiences in large public 
programs. Team support compensates for this to some degree, but background information must be 
available to scaffold learners with the necessary minimal subject matter, collaboration and technical 
resources needed to support exploratory learning. The MDE 663 assessment scheme is heavily 
weighted towards the term's end. This provides too little opportunity for formative assessment earlier 
in the course, with subsequent capacity for remediation and targeted support by the instructor. The 
submission of a team based portal plan and perhaps a subject matter assignment earlier in the course 
would provide this capability. 

For many students working collaboratively is challenging, and the logistics of mediated 
communication increase the risk when the course is offered at a distance. Although our data indicate 
that students do not feel that a face-to-face gathering is critical for success of this design, there is little 
doubt that formal and informal opportunities for both real time and asynchronous interaction are 
important. Students report feeling slightly more stressed by MDE 663's collaborative structure than 
they do most MDE courses. Allan and Lawless (2003) report similar findings, saying that the students 
they studied want to meet the perceived expectations of other group members, have time conflicts 
and/or spend more time on work than anticipated and have to wait on others' work before they can do 
theirs. Most MDE 663 groups indicated communication or collaboration breakdown at some point 
during the development of their projects. 

New research questions spring from this course and subsequent investigation project. For example, 
would the inclusion of more background content, group dynamics and process skills, and technical 
trouble-shooting resources at the beginning of the course help develop more cohesive groups? How 
can changes in the timing and activities of course assignments affect group organization and 
instructor opportunities for remediation? What happens if the instructor imposes more control over 
the structure of groups by, for example, asking teams to choose a leader and/or submit plans for the 
project soon after they begin working on it? Do such impositions reduce the amount of flexible 
learning to a point where the course no longer fits Collis and Moonen's, 2001, Quadrant IV or 
Grabinger and Dunlap's, 2001, REALs? Or should we offer this scaffolding until students are more 
experienced with learning by this pedagogy? Would inclusion of a 'frequently asked questions' section 
on the LMS site improve student autonomy and reduce instructor response time enough to warrant 
the time needed to create and update it? How much would any of these changes positively alter 
student perceptions of learning efficacy? 



This research project shows that MDE 663 emulates the ideals of a constructivist paradigm, and 
typifies Collis and Moonen's, 2001, futuristic flexibility-activity Quadrant IV framework. MDE 663 
students come from Quadrant I course backgrounds, modeling the skills and profiles of learners who 
are masters at knowledge acquisition. Yet, despite the fact that few report having worked in 
collaborative online groups, they rate small group learning almost as high as they do independent 
learning. This project also shows that, with little instructor direction or interference, most students at 
this level demonstrate the necessary individual and group skills of an emerging Quadrant IV learning 
community. 



References 


Allan, J. & Lawless, N. (2003) Stress caused by on-line collaboration in e-learning. In 

Learning Technology Newsletter 5 (4). Retrieved January 15, 2004 from: 

http://lttf.ieee.Org/learn_tech/issues/october2003/index.html#6. 

Berge, Z., & Muilenburg, L. (2000). Barriers to distance education as perceived by managers and 
administrators: Results of a survey. In Clay M. (Ed.), Distance Learning Administration Annual 
2000. Retrieved May 21, 2001 from: http://www.gl.umbc.edu/~berge/man_admin.html 

Campbell, K. (2003). E-Ffective Writing for E-Learning Environments. Hershey, PA: InfoSci. 

Collis, B. & Moonen, J. (2001), . Flexible Learning in a Digital World: Experiences 
and Expectations. London: Koogan Page. 

Dopper, S. & Dijkman, B. (1997). Action learning appropriate for part-time students. In Research in 
Education, 3. 57-59. 

Duffy, T., & Kirkley, J. (2004). Learner-centered theory and practice in distance education: Cases 
from higher education. Mahwah, New Jersey: Lawrence Erlbaum. 

Grabinger R. S. & Dunlap J.C. (2001). Applying the REAL model to web based instruction. Retrieved 
January 31, 2004 from: 

http://ceo.cudenver.edu/~scott_grabinger/downloads/REALWeb.v5.ALTC.pdf 

Kearsley, G. & Schneiderman, G. (1998). Engagement theory: a framework for technology-based 
teaching and learning. In Educational Technology, 38 (5). 20-24. 

Keegan, D. (1996). Foundations of Distance Education - 3rd Edition. London and New York: 
Routledge. 

Lazarus, B. (2003). Teaching courses online: How much time does it take? In Journal of 
Asynchronous Learning, 7(3). Retrieved Dec. 1, 2003 from: 

http://www.aln.org/publications/jaln/v7n3/v7n3_lazarus.asp 

National Science Foundation (1996). Shaping the Future: New Expectations for Undergraduate 
Education in Science, Mathematics, Engineering, and Technology. Washington, DC. 

Ngeow, K. Y. (1998). Enhancing student thinking through collaborative learning. 

ERIC Clearinghouse on Reading English and Communication, (ERIC Document No. 
ED422586). Retrieved January 14, 2004 from: 

http://www.ericfacility.net/ericdigests/ed422586.html. 

Rourke, L. & Anderson, T. (2002) Using peer teams to lead online discussions. In Journal of 
Interactive Media in Education (1). Retrieved November 10, 2003 from: 

http://www.jime.open.ac.Uk/2002/1 . 

Seels, B. (1997). Theory development in educational/instructional technology. In Education 
Technology, 37(1). 3-5. 

Sfard, A. (1998). On two metaphors for learning and the dangers of choosing just one. In 
Educational Researcher, 21 (2). 4-13. 

Springer, L., Stanne, M., & Donovan, S. (1997). Measuring the success of small-group 

learning in college-level SMET teaching: A meta-analysis. Retrieved January 8, 2004 from: 
http://www.wcer.wisc.edu/nise/CL1/CL/resource/scismet.htm 



